Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Templatize event time in metadata.json #2946

Merged
merged 2 commits into from
Oct 24, 2024

Conversation

wslulciuc
Copy link
Member

This PR replaces metadata.json with metadata.template.json. Using the placeholders {{RUN_START_TIME}} and {{RUN_END_TIME_AFTER_*_MINUTES}}, the metadata used to seed the Marquez HTTP server will use the current time as run start time and a random run end time between 5 - 10 minutes.

Signed-off-by: Willy Lulciuc <willy.lulciuc@gmail.com>
@boring-cyborg boring-cyborg bot added the docker label Oct 24, 2024
Copy link

netlify bot commented Oct 24, 2024

Deploy Preview for peppy-sprite-186812 canceled.

Name Link
🔨 Latest commit 9c1e5be
🔍 Latest deploy log https://app.netlify.com/sites/peppy-sprite-186812/deploys/6719fac1e059710008db6060

Signed-off-by: Willy Lulciuc <willy.lulciuc@gmail.com>
Copy link

codecov bot commented Oct 24, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 81.16%. Comparing base (f444458) to head (9c1e5be).
Report is 1 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff            @@
##               main    #2946   +/-   ##
=========================================
  Coverage     81.16%   81.16%           
  Complexity     1506     1506           
=========================================
  Files           268      268           
  Lines          7363     7363           
  Branches        329      329           
=========================================
  Hits           5976     5976           
  Misses         1226     1226           
  Partials        161      161           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Member

@davidsharp7 davidsharp7 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This makes the toy dataset way more useable!

@wslulciuc wslulciuc merged commit f7a9643 into main Oct 24, 2024
16 checks passed
@wslulciuc wslulciuc deleted the feature/templatize-event-time-in-metadata.json branch October 24, 2024 08:16
lmassaoy pushed a commit to nubank/NuMarquez that referenced this pull request Oct 28, 2024

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
* Fixing data quality display. (MarquezProject#2937)

Signed-off-by: phixMe <peter.hicks@pdtechsolutions.com>

* Dataset Version call simplification (MarquezProject#2938)

* Fixing data quality display.

Signed-off-by: phixMe <peter.hicks@pdtechsolutions.com>

* Fixing dataset version calls.

Signed-off-by: phixMe <peter.hicks@pdtechsolutions.com>

---------

Signed-off-by: phixMe <peter.hicks@pdtechsolutions.com>

* feat: allow db-migrate without version (MarquezProject#2936)

Signed-off-by: David Goss <david@davidgoss.co>

* Display full `runID` and check icon when copied (MarquezProject#2940)

Signed-off-by: Willy Lulciuc <willy.lulciuc@gmail.com>

* Deferred copy revert. (MarquezProject#2941)

Signed-off-by: phixMe <peter.hicks@pdtechsolutions.com>

* Long text handling (MarquezProject#2942)

* Deferred copy revert.

Signed-off-by: phixMe <peter.hicks@pdtechsolutions.com>

* Long text handling.

Signed-off-by: phixMe <peter.hicks@pdtechsolutions.com>

* Adding search back in.

Signed-off-by: phixMe <peter.hicks@pdtechsolutions.com>

---------

Signed-off-by: phixMe <peter.hicks@pdtechsolutions.com>

* Use project root for docker volume prefix (MarquezProject#2943)

Signed-off-by: Willy Lulciuc <willy.lulciuc@gmail.com>

* fix: Correct SQL query pagination for DatasetVersion findAll method (MarquezProject#2945)

Signed-off-by: Alper İnan <kurumsal.alper@gmail.com>
Signed-off-by: Alper <kurumsal.alper@gmail.com>

* Update changelog for `0.50.0`

Signed-off-by: Willy Lulciuc <willy.lulciuc@gmail.com>

* Replace `redoc-cli` with `redocly`

Signed-off-by: Willy Lulciuc <willy.lulciuc@gmail.com>

* Prepare for release 0.50.0

Signed-off-by: Willy Lulciuc <willy.lulciuc@gmail.com>

* Prepare next development version 0.51.0-SNAPSHOT

Signed-off-by: Willy Lulciuc <willy.lulciuc@gmail.com>

* Templatize event time in `metadata.json` (MarquezProject#2946)

* Templatize event time in `metadata.json`

Signed-off-by: Willy Lulciuc <willy.lulciuc@gmail.com>

* Use `metadata.template.json`

Signed-off-by: Willy Lulciuc <willy.lulciuc@gmail.com>

---------

Signed-off-by: Willy Lulciuc <willy.lulciuc@gmail.com>

* Update CHANGELOG.md

* Update `web/docs/demo.gif` (MarquezProject#2948)

Signed-off-by: Willy Lulciuc <willy.lulciuc@gmail.com>

* fix(deps): update dependency io.openlineage:openlineage-java to v1.23.0 (MarquezProject#2907)

Signed-off-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

* fix(deps): update dependency org.assertj:assertj-core to v3.26.3 (MarquezProject#2909)

Signed-off-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: Willy Lulciuc <willy@datakin.com>

* fix(deps): update dependency org.postgresql:postgresql to v42.7.4 (MarquezProject#2912)

Signed-off-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

* fix(deps): update dependency org.opensearch.client:opensearch-rest-client to v2.17.1 (MarquezProject#2911)

Signed-off-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: Willy Lulciuc <willy@datakin.com>

* fix(deps): update dependency org.apache.commons:commons-lang3 to v3.17.0 (MarquezProject#2908)

Signed-off-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

* Ignore `**/stats/**` (MarquezProject#2952)

Signed-off-by: Willy Lulciuc <willy.lulciuc@gmail.com>

* Update compatibility for `0.50.0`

* fix(deps): update dependency org.opensearch.client:opensearch-java to v2.16.0 (MarquezProject#2910)

Signed-off-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

* `Dataset.currentVersionUuid` `->` `DatasetVersion.uuid` (MarquezProject#2954)

Signed-off-by: Willy Lulciuc <willy.lulciuc@gmail.com>

* Update Events Page (MarquezProject#2955)

* Tuning the events page for longer events.

Signed-off-by: phixMe <peter.hicks@pdtechsolutions.com>

* Adding events file.

Signed-off-by: phixMe <peter.hicks@pdtechsolutions.com>

* Refetch jobs button.

Signed-off-by: phixMe <peter.hicks@pdtechsolutions.com>

* Refetch jobs button.

Signed-off-by: phixMe <peter.hicks@pdtechsolutions.com>

* Lint

Signed-off-by: phixMe <peter.hicks@pdtechsolutions.com>

---------

Signed-off-by: phixMe <peter.hicks@pdtechsolutions.com>
Co-authored-by: Willy Lulciuc <willy@datakin.com>

* Lineage run attachment issue. (MarquezProject#2953)

Signed-off-by: phixMe <peter.hicks@pdtechsolutions.com>
Co-authored-by: Willy Lulciuc <willy@datakin.com>

* feature: Better handling of missing environment variables in setupProxy.js file. (MarquezProject#2956)

Signed-off-by: Artur Owczarek <owczarek.artur@gmail.com>

---------

Signed-off-by: phixMe <peter.hicks@pdtechsolutions.com>
Signed-off-by: David Goss <david@davidgoss.co>
Signed-off-by: Willy Lulciuc <willy.lulciuc@gmail.com>
Signed-off-by: Alper İnan <kurumsal.alper@gmail.com>
Signed-off-by: Alper <kurumsal.alper@gmail.com>
Signed-off-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Signed-off-by: Artur Owczarek <owczarek.artur@gmail.com>
Co-authored-by: Peter Hicks <phixMe@users.noreply.github.com>
Co-authored-by: davidjgoss <david@davidgoss.co>
Co-authored-by: Willy Lulciuc <willy@datakin.com>
Co-authored-by: Alper İnan <65857758+inanalper@users.noreply.github.com>
Co-authored-by: Willy Lulciuc <willy.lulciuc@gmail.com>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: Artur Owczarek <owczarek.artur@gmail.com>
jonathanpmoraes pushed a commit to nubank/NuMarquez that referenced this pull request Nov 12, 2024

Verified

This commit was signed with the committer’s verified signature.
jonathanpmoraes Jonathan P Moraes
* Fixing data quality display. (MarquezProject#2937)

Signed-off-by: phixMe <peter.hicks@pdtechsolutions.com>

* Dataset Version call simplification (MarquezProject#2938)

* Fixing data quality display.

Signed-off-by: phixMe <peter.hicks@pdtechsolutions.com>

* Fixing dataset version calls.

Signed-off-by: phixMe <peter.hicks@pdtechsolutions.com>

---------

Signed-off-by: phixMe <peter.hicks@pdtechsolutions.com>

* feat: allow db-migrate without version (MarquezProject#2936)

Signed-off-by: David Goss <david@davidgoss.co>

* Display full `runID` and check icon when copied (MarquezProject#2940)

Signed-off-by: Willy Lulciuc <willy.lulciuc@gmail.com>

* Deferred copy revert. (MarquezProject#2941)

Signed-off-by: phixMe <peter.hicks@pdtechsolutions.com>

* Long text handling (MarquezProject#2942)

* Deferred copy revert.

Signed-off-by: phixMe <peter.hicks@pdtechsolutions.com>

* Long text handling.

Signed-off-by: phixMe <peter.hicks@pdtechsolutions.com>

* Adding search back in.

Signed-off-by: phixMe <peter.hicks@pdtechsolutions.com>

---------

Signed-off-by: phixMe <peter.hicks@pdtechsolutions.com>

* Use project root for docker volume prefix (MarquezProject#2943)

Signed-off-by: Willy Lulciuc <willy.lulciuc@gmail.com>

* fix: Correct SQL query pagination for DatasetVersion findAll method (MarquezProject#2945)

Signed-off-by: Alper İnan <kurumsal.alper@gmail.com>
Signed-off-by: Alper <kurumsal.alper@gmail.com>

* Update changelog for `0.50.0`

Signed-off-by: Willy Lulciuc <willy.lulciuc@gmail.com>

* Replace `redoc-cli` with `redocly`

Signed-off-by: Willy Lulciuc <willy.lulciuc@gmail.com>

* Prepare for release 0.50.0

Signed-off-by: Willy Lulciuc <willy.lulciuc@gmail.com>

* Prepare next development version 0.51.0-SNAPSHOT

Signed-off-by: Willy Lulciuc <willy.lulciuc@gmail.com>

* Templatize event time in `metadata.json` (MarquezProject#2946)

* Templatize event time in `metadata.json`

Signed-off-by: Willy Lulciuc <willy.lulciuc@gmail.com>

* Use `metadata.template.json`

Signed-off-by: Willy Lulciuc <willy.lulciuc@gmail.com>

---------

Signed-off-by: Willy Lulciuc <willy.lulciuc@gmail.com>

* Update CHANGELOG.md

* Update `web/docs/demo.gif` (MarquezProject#2948)

Signed-off-by: Willy Lulciuc <willy.lulciuc@gmail.com>

* fix(deps): update dependency io.openlineage:openlineage-java to v1.23.0 (MarquezProject#2907)

Signed-off-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

* fix(deps): update dependency org.assertj:assertj-core to v3.26.3 (MarquezProject#2909)

Signed-off-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: Willy Lulciuc <willy@datakin.com>

* fix(deps): update dependency org.postgresql:postgresql to v42.7.4 (MarquezProject#2912)

Signed-off-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

* fix(deps): update dependency org.opensearch.client:opensearch-rest-client to v2.17.1 (MarquezProject#2911)

Signed-off-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: Willy Lulciuc <willy@datakin.com>

* fix(deps): update dependency org.apache.commons:commons-lang3 to v3.17.0 (MarquezProject#2908)

Signed-off-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

* Ignore `**/stats/**` (MarquezProject#2952)

Signed-off-by: Willy Lulciuc <willy.lulciuc@gmail.com>

* Update compatibility for `0.50.0`

* fix(deps): update dependency org.opensearch.client:opensearch-java to v2.16.0 (MarquezProject#2910)

Signed-off-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

* `Dataset.currentVersionUuid` `->` `DatasetVersion.uuid` (MarquezProject#2954)

Signed-off-by: Willy Lulciuc <willy.lulciuc@gmail.com>

* Update Events Page (MarquezProject#2955)

* Tuning the events page for longer events.

Signed-off-by: phixMe <peter.hicks@pdtechsolutions.com>

* Adding events file.

Signed-off-by: phixMe <peter.hicks@pdtechsolutions.com>

* Refetch jobs button.

Signed-off-by: phixMe <peter.hicks@pdtechsolutions.com>

* Refetch jobs button.

Signed-off-by: phixMe <peter.hicks@pdtechsolutions.com>

* Lint

Signed-off-by: phixMe <peter.hicks@pdtechsolutions.com>

---------

Signed-off-by: phixMe <peter.hicks@pdtechsolutions.com>
Co-authored-by: Willy Lulciuc <willy@datakin.com>

* Lineage run attachment issue. (MarquezProject#2953)

Signed-off-by: phixMe <peter.hicks@pdtechsolutions.com>
Co-authored-by: Willy Lulciuc <willy@datakin.com>

* feature: Better handling of missing environment variables in setupProxy.js file. (MarquezProject#2956)

Signed-off-by: Artur Owczarek <owczarek.artur@gmail.com>

---------

Signed-off-by: phixMe <peter.hicks@pdtechsolutions.com>
Signed-off-by: David Goss <david@davidgoss.co>
Signed-off-by: Willy Lulciuc <willy.lulciuc@gmail.com>
Signed-off-by: Alper İnan <kurumsal.alper@gmail.com>
Signed-off-by: Alper <kurumsal.alper@gmail.com>
Signed-off-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Signed-off-by: Artur Owczarek <owczarek.artur@gmail.com>
Co-authored-by: Peter Hicks <phixMe@users.noreply.github.com>
Co-authored-by: davidjgoss <david@davidgoss.co>
Co-authored-by: Willy Lulciuc <willy@datakin.com>
Co-authored-by: Alper İnan <65857758+inanalper@users.noreply.github.com>
Co-authored-by: Willy Lulciuc <willy.lulciuc@gmail.com>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: Artur Owczarek <owczarek.artur@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

None yet

2 participants